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Executive Summary 



Does providing low-income families vouchers 
or scholarships with which they can select a 
private school improve student achievement? 
The evidence from the Children's Scholarship 
Fund (CSF) program in Charlotte suggests that 
providing low-income families with scholar- 
ships has significant benefits for those families. 
This finding is consistent with the results from 
similar evaluations of scholarship programs in 
New York, Washington, D.C., and Dayton, 
Ohio as well as the results of evaluations of 
publicly funded school choice programs in 
Milwaukee and Cleveland. 

The main findings from this evaluation of the 
Charlotte CSF Program are: 

• Receiving a scholarship to attend private 
school improves scores on standardized math 
tests by between 5.9 and 6.2 national percentile 
ranking points, depending on the type of analy- 
sis performed. 

•Receiving a scholarship to attend a private 
school improves scores on standardized read- 
ing tests by between 5.4 and 7.7 national per- 
centile ranking points, depending on the type 
of analysis performed. 

•Parents were asked to assign their child's 
school a letter grade, A through F. Nearly twice 
as many choice parents gave their child's school 
an A (53%), compared to the public school par- 
ents (26%). Choice parents were also nearly 
twice as likely to report being "very satisfied" 
with virtually all aspects of their children's 
school: location, safety, teaching quality, course 
content, class size, facilities, student respect for 
teachers, information on student progress, reli- 
gious observance, parental support for school, 
discipline, clarity of school goals, teamwork 
among staff, teaching moral values, academic 
quality, and teacher respect for students. 



• Roughly two in five students would give their 
choice school an A compared to 32% of public 
school students. When students were asked how 
they feel about going to school each day, 24% 
of the public school students said that they did 
not want to go compared to 9% of choice stu- 
dents. And 24% of non-scholarship students 
agreed that they did not feel safe at school com- 
pared to 9% of choice students. 

•Parental reports confirm student perceptions 
about safety at school. More than a third of pub- 
lic school parents reported problems with fight- 
ing in school (36%) compared to 16% of choice 
parents. One-quarter of public school parents 
reported problems with racial conflict compared 
to 12% of choice parents. 22% of public school 
parents reported problems with guns or weap- 
ons at their children's elementary schools com- 
pared to 11% of choice parents. And 25% of 
public school parents reported problems with 
destruction of property at school compared to 
12% of choice parents. 

•Because the private schools examined oper- 
ate with far less money per pupil than do the 
public schools, it is not surprising to discover 
that the private schools have more sparse fa- 
cilities and fewer services to offer. For example, 
only 70% of choice parents described their 
school as having a library compared to 90% of 
the public school parents. Only 63% of choice 
parents said that their school had a gym com- 
pared to 91% of public school parents. Only 
71% of choice parents said that their school had 
a cafeteria compared to 89% of public school 
parents. Parents also reported fewer school 
services at the private schools. Only 18% of 
choice parents said that their school had a pro- 
gram for students learning to speak English 
compared to 50% of public school parents. 
Only 49% of choice parents said that their 
school had a program for learning disabilities 



O 

ERIC 



4 



August 2000 



Civic Report 



The Effect of School Choice: An Evaluation of the Charlotte Children's Scholarship Fund Program 



compared to 71% of public school parents, receive some kind of public assistance, such as 
Only 51% of choice parents reported programs food stamps or welfare, while the 1990 census 
for gifted students at their schools compared 



to 72% of public school parents. Choice par- 
ents were also less likely to report that their 
school had a counselor, nurse, music program, 
art program, or prepared lunches. 

The Charlotte CSF Program successfully 
targeted disadvantaged families. In general, 
choice schools were accepting students with 
scholarships who were considerably more 
disadvantaged than typical students in 
Charlotte. Three-quarters of the choice students 
were African-American, while a little more 
than one-third of all students in the Charlotte- 
Mecklenburg school district are African- 
American. As of 1990 the average family 
income in Charlotte was nearly $34,000, almost 
$10,000 more than the average family income 
of choice students 10 years later. Almost one- 
third (32%) of choice families report that they 



reports that only 5% of households in Charlotte 
were on public assistance. And even after one 
year of the scholarship, choice students were 
still scoring well below the national average 
on standardized tests (although they were 
scoring significantly better than they would 
have had they not received the scholarship). 
There is no evidence to support the claim that 
the private schools were "creaming" the best 
students or "dumping" those students whom 
they found undesirable. 

The private schools accepting scholarship stu- 
dents were smaller and had smaller class sizes, 
on average, than the public schools. But small 
class size does not "explain" the higher student 
achievement observed in private schools. Add- 
ing class size to the multivariate model predict- 
ing student test scores shows that class size has 
no effect on student achievement in our sample. 
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Introduction 

Does providing low-income families vouchers 
or scholarships with which they can select a pri- 
vate school improve student achievement? 1 The 
evidence from the Children's Scholarship Fund 
(CSF) program in Charlotte suggests that pro- 
viding low-income families with scholarships 
has significant benefits for those families. This 
finding is consistent with the results from simi- 
lar evaluations of scholarship programs in New 
York, Washington, D.C., and Dayton, Ohio as 
well as the results of evaluations of publicly 
funded school choice programs in Milwaukee 
and Cleveland. The findings of those studies 
have been summarized and discussed else- 
where. 2 This report will focus on presenting the 
results from Charlotte. 

Research Design 

The CSF program offered partial scholarships 
to low-income families in Charlotte with a maxi- 
mum value of $1,700 to attend private schools 
in the 1999-2000 academic year. To ration lim- 
ited funds, scholarships were awarded by lot- 
tery to families that had completed an 
application process. This study examined only 
students enrolled in grades 2 through 8. In that 
age group, 388 students had been awarded 
scholarships by lottery and were enrolled in 
private school, 342 students were not offered 
scholarships by lottery, and 413 students had 
won the lottery to receive a scholarship but did 
not enroll in private school. All of these students 
and their parents were sent invitations to attend 
four testing sessions on a Saturday or Sunday 
between March 18 and April 30, 2000, where 
parents completed surveys while students took 



the Iowa Test of Basic Skills survey version. 
Older students also completed a survey. 

Families whose children were not using schol- 
arships were offered $20 and an opportunity to 
win a new scholarship as incentives to partici- 
pate and to defray the transportation and other 
expenses involved. Families whose children 
were using scholarships were simply asked to 
participate without compensation. Despite these 
relatively modest incentives, our response rate 
was quite good. Of the 1,143 students who were . 
sent invitations to attend a testing session, 452, 
or 40%, participated in the study. The partici- 
pation rate among the students who won the 
lottery and were using scholarships, whom we 
will call "choice students," was 53%. The par- 
ticipation rate among the students who applied 
but failed to win a scholarship in the lottery, 
whom we will call "control students," was 49%. 
The participation rate among the students who 
won a scholarship but did not use it to attend a 
private school, whom we will call "non-com- 
plying students," was 20%. 

Various explanations account for the level of 
participation. The contact information avail- 
able for all students was over a year old. Given 
the high mobility of urban, low-income popu- 
lations, it is likely that many invitations never 
reached their target. In addition, we only of- 
fered four testing opportunities on Saturdays 
or Sundays, which may not have accommo- 
dated the work and social schedules of a num- 
ber of families. Other factors that may have 
influenced participation include transportation 
issues, family motivation, and student coop- 
eration with sacrificing a weekend day to take 
a standardized test. 
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These obstacles to participation were obviously 
most severe among the group that we call non- 
complying students. Many of those families did 
not use the scholarship that was offered to them 
because they moved, exacerbating the difficulty 
of inviting them to participate in this study. 
Other students who were offered scholarships 
but did not use them (and thus did not "com- 
ply" with a lottery research design), may have 
declined the scholarships because they obtained 
access to a desired public school, such as a mag- 
net school or other public school choice pro- 
gram. If these students were doing well in their 
public school they would have little reason to 
participate in the study where the primary in- 
centive was the opportunity to win a private 
school scholarship. Other students may not 
have used a scholarship that was offered be- 
cause they were unable to find a satisfactory 
private school. Yet other students did not use 
a scholarship that was offered because their 
families did not have the financial resources 
to pay the tuition charges above the $1,700 
value of the scholarship. Families that do not 
believe that they will be able to use a new schol- 
arship are unlikely to be enticed by an offer of 
a scholarship to participate in the study. 

Non-compliance and non-participation are is- 
sues in all evaluations, including random-as- 
signment or lottery based studies, such as this 
evaluation and most medical studies. People are 
always free to cease cooperating with research- 
ers and they are always free to refuse the treat- 
ment they are offered. Lotteries in research do 
not ensure identical treatment and control 
groups, but they certainly help get closer to 
achieving comparable groups than other meth- 
ods of selecting subjects. To the extent that non- 
compliance and non-participation produce 
non-identical treatment and control groups, the 
differences can be adjusted statistically with 
little difficulty, as was done in this study. 

Comparability of Groups 

All applicants for scholarships were asked to 
provide their family income at the time of ap- 
plication. More complete demographic informa- 
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tion was collected during the testing sessions, 
but, as noted, not all applicants participated in 
the study. By looking at the income informa- 
tion provided at the time of application we can 
see a number of things: 1) the lottery produced 
two groups that were not significantly differ- 
ent in income (this helps confirm that the lot- 
tery was properly conducted); 2) those 
applicants who participated in the study had 
somewhat higher incomes than those that did 
not; and 3) the differences between the incomes 
of study participants and non-participants are 
roughly equal for lottery winners and lottery 
losers as well as for choice, control, and non- 
complying students. In other words, while those 
who participated in our study differed some- 
what from those who did not, those differences 
do not appear to have biased the comparability 
of our groups. 

The family income of applicants who won the 
lottery to be offered a scholarship was $23,449 
compared to $23,689 for those who lost the lot- 
tery. The difference in income is not statisti- 
cally significant, helping to confirm that the 
lottery was fairly conducted. The family in- 
come of students who participated in the study 
was $25,313, which is significantly different 
from the $22,441 reported at the time of appli- 
cation for those families who did not later par- 
ticipate in testing. This gap of roughly $3,000 
between participants and non-participants ex- 
ists among those who won the lottery (com- 
bining the choice and non-complying students) 
as well as among those who were in the con- 
trol group. Lottery winners who participated 
in the study had average family incomes of 
$25,323, while lottery winners who did not par- 
ticipate had average family incomes of $22,517. 
Control group students who participated in the 
study had average family incomes of $25,297, 
while control group students who did not par- 
ticipate had average family incomes of $22,215. 
Whatever factors influenced participation in 
the study appear to have operated equally on 
lottery winners and lottery losers. 

This claim is further supported by the demo- 
graphic similarity of the treatment and control 
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groups who participated in the study and com- 
pleted our survey. As can be seen in Table 1, 
the lottery winners and losers who participated 
in the study did not differ from each other very 
much in their demographic characteristics. The 
control group had slightly better educated 
mothers, but the difference was not significant, 
while those offered a scholarship were more 
likely to have mothers born outside of the 
United States. Those offered scholarships were 
more likely to receive Supplemental Security 
Income (SSI) for a family disability, while con- 
trol group mothers were more likely to work 
outside of the home. All of these differences 
are modest and we can expect some significant 
differences to be produced by chance when 
comparing a large number of demographic 
characteristics. The overall picture is that de- 
spite non-participation in our study, we man- 



aged to preserve the similarity of the lottery 
winners and losers. 

But we are not primarily interested in compar- 
ing outcomes of lottery winners to lottery los- 
ers. That is, our primary interest is in 
identifying the effect of using a scholarship to 
attend private school, not the effect of being 
offered a scholarship even if one does not use . 
it. We therefore want to compare choice stu- 
dents to the other groups. As can be seen in 
Table 2, choice students differ from the other 
two groups of students (control and non-com- 
plying) whom we are calling "public" students. 
Even though. some of the differences are sta- 
tistically significant, the substantive differences 
are modest. The overall picture is of the choice 
students and comparison groups being quite 
similar, although clearly not identical. 



Table 1 

Demographic Characteristics of Lottery Winners and Losers 




Variable 


Lottery Winners 


Lottery Losers 


Significance 


Mother's Education (11 point scale) 


6.9 


7.8 


0.23 


Mother U.S. Born 


89% 


96% 


0.02 


Attend Religious Services (5 point scale) 


3.3 


3.4 


0.44 


Receive Food Stamps 


19% 


19% 


0.99 


Receive Welfare 


28% 


24% 


0.41 


Receive Social Security 


13% 


11% 


0.44 


Receive Supplemental Security Income 


28% 


15% 


0.01 


Receive HUD Housing Vouchers 


2% 


5% 


0.19 


Family Income (from application) 


$25,323 


$25,297 


0.97 


Family Income (from survey) 


$23,150 


$24,800 


0.14 


Children in Household 


2.4 


2.2 


0.12 


Family Member in Jail 


2% 


1% 


0.88 


Student Has Physical Handicap 


3% 


1% 


0.25 


Student Has Learning Disability 


9% 


11% 


0.55 


Student is a Native English Speaker 


97% 


97% 


0.77 


African-American Mother 


81% 


80% 


0.75 


Two Parent Household 


36% 


33% 


0.53 


Male Student 


49% 


41% 


0.1 


Year of Mothers Birth 


1962 


1963 


0.28 


Mother Employed Full-Time 


60% 


68% 


0.03 


Mother Single, Never Married 


27% 


29% 


0.14 


Mother is Baptist 


38% 


42% 


0.96 


N 


206-267 


135-161 




Significance below .05 is conventionally considered statistically 


significant. 
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Table 2 

Demographic Characteristics of Choice and Public School Students 


Variable 


Choice 


Public 


Significance 


Mother's Education (1 1 point scale) 


7.1 


7.3 


0.75 


Mother U.S. Born 


88% 


94%' 


0.02 


Attend Religious Services (5 point scale) 


3.4 


3.2 


0.08 


Receive Food Stamps 


13% 


22% 


0.03 


Receive Welfare 


6% 


9% 


0.26 


Receive Social Security 


13% 


12% 


0.79 


Receive Supplemental Security Income 


7% 


6% 


0.87 


Receive HUD Housing Vouchers 


1% 


• 5% 


0.07 


Family Income (from application) 


$26,084 


$24,714 


0.24 


Family Income (from survey) 


$23,450 


$23,850 


0.88 


Children in Household 


2.4 


2.2 


0.12 


Family Member in Jail 


1% 


2 % 


0.15 


Student Has Physical Handicap 


3% 


2 % 


0.72 


Student Has Learning Disability 


4% 


13% 


0.00 


Student is a Native English Speaker 


98% 


97% 


0.45 


African-American Mother 


76% 


85% 


0.04 


Two Parent Household 


42% 


29% 


0.01 


Male Student 


49% 


44% 


0.34 


Year of Mother's Birth 


1961 


1963 


0.01 


Mother Employed Full-Time 


51% 


73% 


0.00 


Mother Single, Never Married 


23% 


31% 


0.00 


Mother is Baptist 


33% 


45% 


0.00 


N 


145-189 


197-239 




Significance below .05 is conventionally consi 


dered statistica 


lly significant. 





We employ two strategies in this study for com- 
paring the outcomes of choice students to those 
of the other groups. The first strategy employs 
what is called a quasi-experimental research de- 
sign in which observed differences between the 
groups that are theoretically expected to be re- 
lated to the outcomes are controlled statisti- 
cally. 3 Because the groups are already very 
similar, we have less reason to fear that unob- 
served differences between the groups bias our 
estimates of the effect of using a scholarship. 
Concern about the unobserved differences be- 
tween families that send their children to pub- 
lic and private schools has always limited our 
ability to draw conclusions from comparisons 
of the outcomes of students enrolled in public 
and private schools. Even after controlling for 
observed demographic differences, research- 
ers could always wonder whether unobserved 



differences that were not being controlled sta- 
tistically, such as parental motivation or the 
intellectual richness of home life, actually ac- 
counted for the differences in student out- 
comes instead of the schools. 

In our case, however, the application process 
and lottery have produced groups for compari- 
son that are already quite similar on observed 
as well as (in all likelihood) unobserved charac- 
teristics. All families had to be sufficiently mo- 
tivated to complete an application for a 
scholarship. All families had to be low-income 
to qualify for a scholarship. A lottery was used 
to select who would be offered scholarships, cre- 
ating, as we have confirmed, two groups that 
were nearly identical. While non-compliance 
and non-participation have caused the groups 
we are comparing to stray from being identical 
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in their background characteristics, they are still 
quite similar so that controlling for observed 
characteristics is likely to produce results in 
which we can have high confidence. 

The second strategy to identify the effect of us- 
ing a scholarship is to use the lottery as an "in- 
strument" to estimate who uses the scholarships 
that are offered. 4 That is, we first predict who 
will use a scholarship, using whether someone 
won the lottery to help us make that prediction, 
and then we determine whether the students we 
predict used a scholarship have better outcomes. 
By using the predicted users of scholarships 
rather than the actual users, we remove the bias 
that may be introduced by the fact that the stu- 
dents who used the vouchers may differ (in un- 
observed ways) from the students who were 
offered a voucher but did not use them. Our es- 
timated scholarship users will be nearly identi- 
cal in their background characteristics to the 
groups against which we are comparing them. 
This technique, known as an instrumental analy- 
sis or a two-stage Heckman analysis, is a widely 
used strategy among economists that can pro- 
duce very reliable findings. 

Test Score Outcomes 

Using these two strategies we can estimate the 
benefit of receiving a scholarship to attend a 
private school in Charlotte on student standard- 
ized test scopes after one year. Using the quasi- 
experimental technique, we compute the effect 
of using a scholarship controlling for a host of 
background characteristics, including mother's 
education, mother's race, family income, two- 
parent household, and sex of student. These 
background characteristics are widely thought 
to be strongly related to student achievement 
in education research. 5 We could control for 
additional background characteristics, but we 
would lose additional students from our analy- 
ses due to the fact that not all parents completed 
all questions on their surveys without gaining 
much explanatory power. 

The benefit of receiving a scholarship on stu- 
dents' math scores is 5.9 percentile points at 



the end of the first year (see Table 3). The ben- 
efit of using a scholarship to attend a private 
school on reading scores is 6.5 percentile points 
after one year. Gains in both math and reading 
are statistically significant at the conventional 
p < .05 level. 

When using the instrumental analysis it is ar- 
guable that it is not necessary to control for 
background characteristics because we have re- 
captured the nearly identical comparison 
groups produced by the lottery to award schol- 
arships. The advantage of not controlling for 
any background characteristics is that we avoid 
losing any cases due to missing data from the 
parent surveys. An instrumental analysis with- 
out controlling for any background character- 
istics shows the benefit of using a scholarship 
to be 6.1 percentile points for math and 5.4 per- 
centile points for reading. Both results are sta- 
tistically significant. 

The estimated effect of using a scholarship 
from the instrumental analysis increases some- 
what if we add controls for background char- 
acteristics, although we do lose nearly 100. cases 
because of missing data on one or more vari- 
able. The benefit of receiving a. scholarship on 
math scores in this analysis is 6.2 percentile 
points, while the benefit for reading is 7.7 per- 
centile points. The math effect is statistically 
significant at p < .1 and the reading effect is 
significant at p < .05. 

The test score results across these analyses are 
consistently positive and significant. Having 
access to a private school with a scholarship 
improves student performance on standard- 
ized test scores by between 5.4 and 7.7 per- 
centile points for math and reading after only 
one year's time. On average, a scholarship 
makes the difference between students scor- 
ing in the low 30s and the high 30s. This gain 
is fairly large. Using within sample variance, 
the benefit is approximately .25 standard de- 
viations for math and reading, which educa- 
tion researchers generally consider large. To 
put the gain in perspective, the difference be- 
tween minority and white students nation- 
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Table 3 








The Effect of Attending a Private School with a Scholarship on Test Scores 




The Effect of Attendina a Private School with a ScholarshiD on Math Scores 








Quasi-Experimental 


Instrumental 


Instrumental 








w/o background 


w/ background 








controls 


controls 


Variable 


Effect 


Significance 


Effect Significance 


Effect 


Significance 


Choice 


5.9 


0.04 


6.1 0.01 


6.2 


0.10 


African-American Mother 


-13.5 


0.00 




-13.4 


0.00 


Mothers Education 


2.4 


0.00 




2.4 


0.00 


Family Income (in $5,000 increments) 


2.0 


0.01 




2.0 


0.01 


Two-Parent Household 


2.8 


0.40 




2.6 


0.44 


Male Student 


0.4 


0.86 




0.4 


0.87 


Non-Complying Student 


0.7 


0.85 








Constant 


10.3 


0.09 


29.1 0.00 


10.3 


0.10- 


N 


357 




436 


357 




Adjusted R-Square 


0.14 




0.01 


0.14 




The Effect of Attendina a Private School with a Scholarshio on Reading Scores 








Quasi- 


-Experimental 


Instrumental 


Instrumental 








w/o background 


w/ background 








controls 


controls 


Variable 


Effect 


Significance 


Effect Significance 


Effect 


Significance 


Choice 


6.5 


0.03 


5.4 0.00 


7.7 


0.05 


African-American Mother 


-11.0 


0.00 




-10.7 


0.00 


Mother's Education 


2.8 


0.00 




2.7 


0.00 


Family Income (in $5,000 increments) 


1.6 


0.06 




1.7 


0.05 


Two-Parent Household 


10.0 


0.01 




9.3 


0.01 


Male Student 


-5.7 


0.04 




-5.7 


0.04 


Non-Complying Student 


3.3 


0.42 








Constant 


13.0 


0.04 


34.7 0.04 


12.8 


0.05 


N 


357 




436 


357 




Adjusted R-Square 


0.17 




0.01 


0.17 




Significance below .05 is conventionally 


considered statistically significant. 
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wide is approximately 1 standard deviation. 
The benefits observed from the Charlotte CSF 
program are roughly one-quarter as large at 
the end of the first year. 

Parental and Student Satisfaction 

Another important indicator of the benefit of a 
program on students is how parents describe 
those benefits. While parents' judgments may 
be distorted by the desire to affirm their deci- 
sion, parents are particularly well-positioned to 
assess effects on their own children given how 
much more contextual information they have 
about how their children are doing. According 
to parents, having a scholarship to attend pri- 
vate school is clearly beneficial. Parents were 
asked to assign their child's school a letter grade, 
A through F. Nearly twice as many choice par- 
ents gave their child's school an A (53%), com- 
pared to the public school parents (26%). (See 
Table 4) Choice parents were also much more 



likely to report being "very satisfied" with vir- 
tually all aspects of their children's school: lo- 
cation, safety, teaching quality, course content, 
class size, facilities, student respect for teach- 
ers, information on student progress, religious 
observance, parental support for school, disci- 
pline, clarity of school goals, teamwork among 
staff, teaching moral values, academic quality, 
and teacher respect for students. 

The older students who completed a survey 
during . the testing sessions similarly reported 
significantly more positive assessments of their 
private school than did those students who did 
not receive a scholarship. Roughly two in five 
students would give their choice school an A 
compared to 32% of public school students. (See 
Table 5) When students were asked how they 
feel about going to school each day 24% of the 
public school students said that they did not 
want to go compared to 9% of choice students. 
And 24% of non-scholarship students agreed 



Table 4 

Parental Satisfaction 



Variable 


Choice 


Public 


Significance 


Would Give School an A 


53% 


26% 


0.00 


Percentage Very Satisfied With... * 
School Location 


47% 


29% 


0.00 


School Safety 


58% 


32% 


0.00 


Teaching Quality 


54% 


27% 


0.00 


What is Taught 


64% 


33% 


0.00 


Class Size 


61% 


24% 


0.00 


Facilities 


53% 


25% 


0.00 


Students Respect Teachers 


61% 


31% 


0.00 


Information on Student Progress 


60% 


29% 


0.00 


Observe Religion 


65% 


25% 


0.00 


Parental Support for School 


58% 


27% 


0.00 


Discipline 


53% 


30% 


0.00 


Clarity of School Goals 


50% 


25% 


0.00 


Teamwork Among Staff 


54% 


26% 


0.00 


Teaching Values 


62% 


27% 


0.00 


Academic Quality 


55% 


27% 


0.00 


Teachers Respect Students 


58% 


26% 


0.00 


N 


185-190 


231-242 





Significance below .05 is conventionally considered statistically significant. 
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Table 5 

Student Assessments of Schools 



Variable 


Choice 


Public 


Significance 


Would Give School an A 


40% 


32% 


0.05 


Do Not Want to Go to School 


9% 


24% 


0.00 


Do Not Feel Safe at School 


9% 


24% 


0.03 


Strongly Agree that. . . 

Teachers are Interested in Students 


52% 


28% 


0.00 


Teachers Listen 


44% 


26% 


0.01 


Teaches are Fair 


35% 


22% 


0.00 


Agree that... 

Students Get Along with Teachers 


66% 


38% 


0.00 


N 


96-98 


107-109 





Significance below .05 is conventionally considered statistically significant. 



that they did not feel safe at school compared 
to 9% of choice students. 

Parental reports confirm student perceptions 
about safety at school. More than a third of pub- 
lic school parents reported problems with fight- 
ing in school (36%) compared to 16% of choice 
parents. (See Table 6) One-quarter of public 
school parents reported problems with racial 
conflict compared to 12% of choice parents. 22% 
of pub He school parents reported problems with 
guns or weapons at their children's elementary 



schools compared to 11% of choice parents. And 
25% of pub He school parents reported problems 
with destruction of property at school compared 
to 12% of choice parents. 

School Facilities and Services 

Given the overwhelmingly positive description 
of the choice schools and given the test score 
improvements, one might expect that the pri- 
vate schools are simply more luxurious schools 
with better resources. Far from it. Most of the 



Table 6 

Parent Description of School: 

Percentage Reporting Problems are Somewhat or Very Serious 



Variable 


Choice 


Public 


Significance 


Fighting 


16% 


36% 


0.00 


Racial Conflict 


12% 


25% 


0.00 


Guns or Weapons at School 


11% 


22% 


0.00 


Destroying Property 


12% 


25% 


0.00 


Cheating 


16% 


36% 


0.00 


Cutting Classes 


15% 


26% 


0.01 


Tardiness 


23% 


33% 


0.08 


N 


185-188 


233-238 





Significance below .05 is conventionally considered statistically significant. 
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private schools at which students used scholar- 
ships operate on nearly half as much money per 
pupil as do the public schools. Tuition at most 
of the private schools is well below $3,000 and 
additional fundraising brings no more than a 
few hundred dollars per student. 

With far less money it is not surprising to dis- 
cover that the private schools have more sparse 
facilities and fewer services to offer. For ex- 
ample, only 70% of choice parents described 
their school as having a library compared to 90% 
of the public school parents. (See Table 7) Only 
63% of choice parents said that their school had 
a gym compared to 91% of public school par- 
ents. Only 71% of choice parents said that their 
school had a cafeteria compared to 89% of pub- 
lic school parents. Parents also reported fewer 
school services at the private schools. Only 18% 
of choice parents said that their school had a 
program for students learning to speak English 
compared to 50% of public school parents. Only 
49% of choice parents said that their school had 
a program for learning disabilities compared to 
71% of public school p arents. Only 51% of 
choice parents reported program for gifted stu- 
dents at their schools compared to 72% of pub- 



lic school parents. Choice parents were also less 
likely to report that their school had a counse- 
lor, nurse, music program, art program, and 
prepared lunches. 

There were some things that were equally or 
more available at choice schools. For example, 
choice and public schools were roughly equally 
likely to have a computer lab. And choice 
schools were equally likely to offer individual 
tutors and more likely to offer after-school pro- 
grams. When parents report that they are more 
satisfied with the choice school facilities, they 
clearly must be focusing on these features that 
they believe are more important. Choice schools 
appear to have far fewer resources but to con- 
centrate those resources on providing the facili- 
ties and services that parents value most. 

What Might Account for Choice 
School Success? 

If the private schools are not better funded and 
do not have nicer facilities and services by ob- 
jective standards, why do parents like them so 
much? The most obvious answer is that parents 
like the choice schools because their children are 




Table 7 

Parent Description of School Facilities and Services 



Variable 


Choice 


Public 


Significance 


Computer Lab 


80% 


85% 


0.23 


Library 


67% 


90% 


0.00 


Gym 


63% 


91% 


0.00 


Cafeteria 


71% 


89% 


0.00 


Program for Non-English Speakers 


18% 


50% 


0.00 


Individual Tutors 


64% 


64% 


0.92 


Program for Learning Disabilities 


49% 


71% 


0.00 


Program for Gifted Students 


51% 


72% 


0.00 


School Counselor 


66% 


83% 


0.00 


Nurse 


46% 


79% 


0.00 


Music Program 


85% 


93% 


0.01 


Art Program 


68% 


79% 


0.02 


After-School Program 


92% 


83%. 


0.00 


Prepared Lunch 


74% 


92% 


0.00 


N 


125-185 


130-240 





Significance below .05 is conventionally considered statistically significant. 
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learning more. But what might account for this 
better student achievement? While this study 
is not designed to address this question fully, 
it is possible to speculate based on the evidence 
that was collected. Some of the most impor- 
tant differences between the choice and public 
schools pertain to the quality and motivation 
of teachers in the two sectors. As we have al- 
ready seen, parents give very strong marks to 
the quality of instruction at the choice schools. 
Interestingly, so do the students. Students are 
almost twice as likely to report that teachers at 
choice schools are "interested in students" than 
are public school students. (See Table 5 on page 
10) Choice students are also significantly more 
likely to report that their teachers listen to stu- 
dents, that teachers are fair, and that students 
get along with teachers. 

Despite having less money for teacher salaries 
and benefits, private schools appear to be bet- 
ter able to recruit quality teachers and dismiss 
bad ones. They may attract more quality teach- 
ers because they can offer positive working 
conditions, an organization with a clear sense 
of mission, and greater autonomy in the class- 
room. Layers of bureaucratic regulations and 
control in the public schools, perhaps a by- 
product of political governance of the schools, 
makes it difficult for public schools to main- 
tain positive working conditions, agree on a 
clear mission, or provide autonomy in the 
classroom. Importantly, school district and 
union rules also make the removal of bad teach- 
ers much more difficult in public schools than 
in private schools. 

Choice and public schools also differ in their 
overall size and in their average class size. The 
median choice student is enrolled in a school 
that has between 151 and 300 students. The 
median public school student is in a school that 
has between 451 and 600 students. The median 
choice student is in a class that has between 11 
and 15 students, while the median public 
school student is in a class with between 21 and 
25 students. Education researchers are increas- 
ingly recognizing that there may be 
diseconomies of scale in education. 6 That is, 



smaller school districts tend to do better than 
larger school districts, smaller school buildings 
tend to do better than larger school buildings, 
and smaller classrooms may do better than 
larger classrooms. Smallness may permit the 
development of a sense of community and 
common purpose, which may be key to school 
success. And smallness obviates the need for 
rigid rules that restrict the autonomy of prin- 
cipals and teachers. 

Some critics of school choice suggest that small 
classes in private schools "explain" the achieve- 
ment benefits of voucher and private scholar- 
ship programs. If only public schools were 
provided with additional resources to reduce 
class size, they too could improve achievement. 
This, of course, begs the question: why have the 
private schools with fewer resources been able 
to produce significantly smaller classes than 
public schools? And what assurance is there that 
additional funds for public schools will lead to 
reduced class size and not to higher paid teach- 
ers or more non-teaching staff? 

Interestingly, adding class size to the model in 
Table 3 that estimates student achievement 
shows that class size is not significantly related 
to student achievement in our sample. In other 
words, class size does not "explain" the achieve- 
ment benefits of receiving a scholarship to at- 
tend private school in Charlotte. 

In addition, one should not attempt to explain 
ivhy private schools appear to outperform pub- 
lic schools while attempting to estimate whether 
private schools outperform public schools. By 
analogy, if we want to know whether the Cubs 
or Yankees are better baseball teams, we should 
not control for pitching, hitting, and fielding. 
Pitching, hitting, and fielding may help explain 
why one team is better than another, but they 
should not be considered when assessing 
whether one team is better than another. Simi- 
larly, when we are addressing whether students 
do better when they have access to a private 
school with a scholarship we should not attempt 
to control for those factors that may help explain 
why private schools may be better. 
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Creaming and Dumping 

Another prominent explanation for private 
school success is that private schools are able to 
select their students by skimming off the cream 
of students and dumping the undesirable stu- 
dents. In truth, public schools can also be selec- 
tive. Some magnet and other public school 
programs have academic or racial criteria for ad- 
mission. And students whom the public schools 
decide they cannot educate properly are some- 
times sent to other public schools or to private 
schools at public expense. Not every public 
school is obligated to accept every student. 

In our sample we saw little evidence to suggest 
that private schools were creaming the best stu- 
dents and dumping the worst. First, almost no 
private schools were administering admissions 
tests to select academically advantaged stu- 
dents. Families who were unable to get their 
children into the schools they desired were 
asked to provide the reasons for their inability 
to gain access to those schools. More than three- 
fifths of these families cited financial constraints 
as blocking their access to a desired school. Ac- 
cording to parental reports only two students 
out of all of the students offered a scholarship 
failed to gain admission to a private school be- 
cause of an admissions test. 

Second, there is no evidence that private schools 
expelled undesirable students or asked them not 
to return. Parents of students who did not com- 
plete the year at the same private school were 
asked to describe the reason for their switch. Not 
one reported that they switched schools because 
their child was expelled. And of those parents 
who reported that they might not return to the 
same school next year not one reported that their 
child was asked not to return. In short, there is 
virtually no evidence that the choice schools 
academically screened their students for admis- 
sion or expelled or "counseled out" students 
they found undesirable. 

Parents were also asked whether their children 
had any physical handicaps, learning disabili- 
ties, or issues learning to speak English. Very 

O 

ERIC 



few reported physical handicaps, only 3% of 
choice students and 2% of public school par- 
ents. Similarly low percentages of choice and 
public school parents reported that English was 
not their child's native language. However, 
choice parents reported fewer children with 
learning disabilities (4%) than did public school 
parents (13%). As we have already observed, 
given their lower level of funding fewer pri- 
vate schools offer special programs for learn- 
ing disabilities. This difference in the 
percentage of students with learning disabili- 
ties may also be partially explained by differ- 
ing incentives in the public and private schools 
to label students as having learning disabili- 
ties. Public schools obtain additional resources 
for students labeled as learning-disabled and 
may be able to exempt learning-disabled stu- 
dents from accountability testing. 

While this difference in learning disabled stu- 
dents at choice and public schools is signifi- 
cant, it is not necessarily evidence of creaming 
or dumping. It may be evidence of parental 
choice. Parents of children with special needs 
are more likely to choose schools that have 
additional funds to offer programs that address 
those special needs. A fair test of whether pri- 
vate schools are avoiding learning disabled stu- 
dents would compare the rates of learning 
disabilities when private schools are given the 
same additional resources to serve those chil- 
dren as the public schools receive. In the ab- 
sence of such a test, this evidence on learning 
disabilities is ambiguous. 

In general, choice schools were accepting stu- 
dents with scholarships who were considerably 
more disadvantaged than typical students in 
Charlotte. Three-quarters of the choice students 
were African-American, while a little more than 
one-third of all students in the Charlotte- 
Mecklenburg school district are African-Ameri- 
can. 7 As of 1990 the average family income in 
Charlotte was nearly $34,000, almost $10,000 
more than the average family income of choice 
students 10 years later. Almost one-third (32%) 
of choice families report that they receive some 
kind of public assistance, such as food stamps 
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or welfare, while the 1990 census reports that 
only 5% of households in Charlotte were on 
public assistance. And even after one year of the 
scholarship, choice students were still scoring 
well below the national average on standard- 
ized tests (although they were scoring signifi- 
cantly better than they would have had they not 
received the scholarship). 

It takes some doing to suggest that the schol- 
arship families that enrolled in private school 
are the cream when those families are more 
likely to be African-American, low income, on 
public assistance, and score below-average on 
test scores than typical families in Charlotte or 
the United States. It is clear that the CSF pro- 
gram in Charlotte is successfully targeting dis- 
advantaged students. While it may not reach 
the most severely disadvantaged, just as Food 
Stamps or housing vouchers do not always 
reach the most disadvantaged, the scholarship 
program is clearly offering opportunities to 
families that lack them. And it is also clear that 
the private schools are taking on these disad- 
vantaged students, not creaming off the best 
and dumping the worst. 

Implications for School Choice Policies 

The privately-funded scholarship program in 
Charlotte differs from what a publicly-funded 
school choice program would likely be in a num- 
ber of respects. First, the scholarship had a low 
monetary value and always required a signifi- 
cant co-payment from the family toward tuition. 
A publicly-funded voucher would likely be 
worth considerably more money and would 
require little if any co-payment from the receiv- 
ing families. This difference may alter the ben- 
efit we would expect to see from gaining access 
to private schools. The additional money a pub- 
licly-funded voucher would provide to private 



schools might increase the expected benefit, but 
the reduced co-payment from families might 
alter the characteristics of participating families 
and reduce the benefit. 

Second, privately-funded scholarships place 
little or no regulation on the activities of pri- 
vate schools, while publicly-funded vouchers 
would likely carry with them more regulation. 
That regulation might improve the benefits of 
the program by ensuring equal access and the 
provision of consumer information, but regula- 
tion might also reduce the benefits of the pro- 
gram by encumbering schools. 

Third, the CSF scholarship program was small 
enough so that its recipients could be accom- 
modated by spare capacity in existing private 
schools. A larger, publicly-funded school choice 
program would require the addition of new 
private schools. The outcomes in new private 
schools might be better or worse than that ob- 
served in existing schools. 

There is no way of addressing these issues fully 
without attempting additional publicly-funded 
programs on a larger scale. The results from the 
evaluation of the Charlotte CSF scholarship pro- 
gram strongly suggest that attempting larger- 
scale, publicly-funded programs is desirable. 
The positive findings from Charlotte are con- 
sistent with positive results from evaluations of 
privately-funded programs in New York, Wash- 
ington, D.C., and Dayton as well as pilot, pub- 
lic choice programs in Milwaukee and 
Cleveland. Whether those positive results will 
hold when school choice is attempted in a more 
complete way cannot be known at present. The 
existing evidence is encouraging enough that we 
should implement new school choice programs 
to see if these significant benefits can be repro- 
duced on a larger scale. 
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